1/18 

FIG. 1 



Initiator SIDE 
AP1 HCA1 



Target SIDE 
HCA2 AP2 



START OF BATCH 
TRANSFER 



1 01 — h transfer request! 

102 — [transfer request 1 

103 — (transfer request! 
105— [transfer request i 




BA- 
RE 



351 



BATCH TRANSFER COMPL. 
STATUS 



CH TRANSFER 
RY 



352 



BATCH TRANSFER COMPL. 
STATUS 



BATCH TRANSFER 
TERMINATION 






301 



COMPL STATUS 



701 



PROCESS 1 STARTS! 



CONTROL BY BATCH TRANSFER 
CONDITION 



312 



= - COMPL. STATUS 



712 



PROCESS 2 STARTSl 



-di COMPL, STATUS 

~ : 

313 



PROCESS 3 STARTSl 
s 



713 



- - I COMPL STATUS 



315 



PROCESS n STARTSl 
^ 



715 



2/18 



FIG. 2 



Initiator SIDE 
AP1 HCA1 



Target SIDE 
HCA2 AP2 



Send 1 
121— (transfer request I 




Sendl 
(RETRY) 



361 — [TRANSFER STATUS 1 



Send n 
125 Transfer request! 



3fi5- TTRANSFER STATUS! 




NAK 




ACK 





321 



COMPL. STATUS 



721 

PROCESS 1 STARTSl 



325 



l: 725 

COMPL. STATUS [pR 0CESS „ STARTS I 



IN A CONVENTIONAL METHOD OF DATA 
TRANSFER (SINGLE DATA TRANSFER BASE), 
OVERHEADS OCCUR BEFORE AND AFTER 
EVERY DATA TRANSFER AND THIS IS A 
BOTTLENECK OF INCREASING EFFICIENCY 
OF TRANSFER. 



3/18 



FIG. 3 



Initiator SIDE 
AP1 HCA1 

RDMA SEND1 
131 — ITRANSFER REQUEST 1 



Target SIDE 
HCA2 AP2 





RDMA SEND1 
(RETRY) 



371 

I TRANSFER COMPL STATUS] 



ERROR 

sis** 8 * 




COMPL STATUS 



331 



PROCESS 1 STARTSl 
^ 



731 



IN A CONVENTIONAL METHOD OF DATA 
TRANSFER (SERIAL DATA TRANSFER BASE), 
ONLY AFTER TRANSFER OF ALL SERIAL 
DATA IS COMPLETED WITHOUT ERRORS, A 
PROCESS FOR THE DATA STARTS. 



4/18 



FIG. 4 



31 

1 



CQ 



61 



PROCESS 



51 



PROCESS CODE 



,41 



21 



22 



QP 






SQ 




RQ 




I 


11 



QP 

SQ 




RQ 




i 


12 



J2± 



81 



RECORD 
BUFFER 



81 



RECORD 
BUFFER 



PROCESS BUFFER 



-71 



53 



PROCESS 




CQ 



T 
32 



QP 
43 



23 44 24 62 

/ — 



PROCESS BUFFER 

,82 



RECORD 
BUFFER 



82 



RECORD 
BUFFER 



PROCESS CODE 



PROCESS 



52 



PROCESS 



54 



5/18 



FIG. 5 



RX PORT 



631 



632 



RE 



-613 
633 



623 



RX PORT 



643 



650 



670 



INTERNAL 
BUS 



MEMORY 



660 



CONNECTION INTERFACE 



641 



RL 




RP 




TP 




TL 



642 



TE 



6/18 



FIG. 6 



HCA1 HCA2 




PACKET 
411 



PACKET 
413 
J 



PACKET 
414 

_J 



\ » 
\ \ 

X I 

\ \ 




/ / 
i i 
i i 
i i 
i i 
i i 


\ \ 
\ \ 
\ 




i i 
i i 


I 


I I 


Q £ RECORD BUFFER \ 
PROCESS BUFFER ^ 




7/18 



FIG. 7 



AREA, SOURCE OF 
RMDA TRANSFER 
85 




86 
AREA, 

DESTINATION 
OF RMDA 
TRANSFER 



8/18 



FIG. 8 



TRANSFER REQUEST 
^100 



CQ STATUS 

.,300 



id 



code 



Add 
ress 



length 



SQ 

TRANSFER REQUEST 



^105 


si 


BATCH 

Dma 

send 


Add_si 


Len_si 


TRANSFER REQUEST 
^101 


s1 


BATCH 
send 


Add_s1 


Len_s1 



39 



BATCH 
LTRANSFER 
REQUESTS 
ON SQ 



Add si 





sid 


Status 




CQ 




si 


St_si 




305 






^301 








S1 


St_s1 




,87 



AREA, SOURCE OF RDMA TRANSFER 



Add_s1 ^89 
RECORD BUFFER! 
Add s1 + len s1 



Add si + len si 



APPLICATION MEMORY SPACE 



9/18 



FIG. 9 



r 



411 

^_ 

ROUTING 
HEADER 



442 

S 

TRANSPORT 
HEADER 



443 

\ 

PAYLOAD 



^ 

440 

PACKET 



444 

CRC 



< 



(a) 

EX. WHERE BATCH 
TRANSFER MODE 
INFO IS INCLUDED 
IN TRANSPORT 
HEADER 

(b) 

EX. WHERE BATCH 
TRANSFER MODE 
INFO IS INCLUDED 
IN PAYLOAD 



PRIORITY 


DESTINATION ADDRESS 


PACKET LENGTH 




SOURCE ADDRESS 


Opcode 




BATCH TRANSFER FLAG: 




DESTINATION QP 


BATCH TRANSFER CON0. FIELD 

VSSMSS/SSSSSSSSSSSS7SSSSSSSSSSSSJ 


) PSN 


^451 v 450 



I ROUTING 
J HEADER 



TRANSPORT 
HEADER 



RECORDS° F BATCH TRANSFER 


BATCH TRANSFER COND. FIELD 


, 455 


»p 

456 





^PAYLOAD 



>CRC 



00: INITIAL RECORD OF BATCH TRANSFER 
01: MIDDLE RECORD OF BATCH TRANSFER 
10: LAST RECORD OF BATCH TRANSFER 
11: CANCEL RECORD OF BATCH TRANSFER 



(c) 

) BATCH TRANSFER 
FLAG 





OJ 




cz 




£=1 


£=> 




Q 




cn 


DC 




DC 




O 


O 




O 




LU 


CD 
LU 




<J> 
LU 




DC 


DC 




DC 





(d) 

BATCH TRANSFER 
COND. FIELD 



V 



NUMBER OF BATCH 
TRANSFER RECORDS 



10/18 



FIG. 10 



( REQUEST FROM Initiator") 



J2± 



1001 



BATCH TRANSFER 
MODE START REQUEST 



req 



ack 



( RESPONSE FROM TargeQ 



1002 



BATCH TRANSFER MODE 
START RESPONSE 




,1015. 



,1017 



SEND 




DMA S 


RES 




Res 





cancel 




BATCH TRANSFER MODE 
TERMINATION RESPONSE 


req 


BATCH TRANSFER MODE 
TERMINATION REQUEST 




ack 













C END ) 



11/18 



FIG. 11 



(SEND REQUEST) 




Cancel 
REQUEST 



1 


OK 

r 


SENDING 







1201 




1211 



Cancel RESPONSE 



1212 



Cancel REQUEST 
(OPTION) 



1221 



SEND STATUS 
LOGGING 



( END ) 



12/18 



FIG. 12 



(SEND RESPONSE) 



Cancel 
REQUEST 



i c± 



1111 



Cancel RESPONSE 






OK 


RECEIVING 







1101 




YES 



1110 



COMPL. STATUS IS 
POSTED TO CQ 



V r± 



1112 



Cancel REQUEST 
(OPTION) 



1121 



RECEIVE STATUS 
LOGGING 



( END ) 



13/18 



FIG. 13 



HOST COMPUTER 



560 



HOST INTERFACE 
NETWORK 
501 




^7 
502 

DISK INTERFACE 
NETWORK 



14/18 



FIG. 14 



HOST 

CHANNEL 

INTERFACE 



511 513 



511 



511 



511 



511 



511 



511 



511 



MEMORY 



512 



cr 

CO 
GO 



MEMORY 



513 



512 



O 

CO 
CO 
LU 

CJ> 

o 
cr 



HOST HUB 



o 

CO 
CO 
LU 
CJ> 
O 



O 
CO 
CO 



513 



MEMORY 



512 



MEMORY 



603 



514 



512 



513 



603 



HCA 



HCA 



15/18 



FIG. 15 



DISK 

CHANNEL 
INTERFACE 



521 523 



521 



521 



521 



521 



zr 



521 



521 



521 



525 



MEMORY 



522 



MEMORY 



CD 
co 

CO 
l_LJ 

CD 
CD 



DC 
CD 
CO 
CO 
LCI 
CD 
CD 



523 



522 



DISK HUB 



524 



CACHE MEMORY 



604 



604 



HCA 



HCA 



16/18 



FIG. 16 



560 



HOST COMPUTER 




HOST INTERFACE 
NETWORK 
501 



DISK CONTROLLER 
INTERNAL NETWORK 
503 



580 i 
MEMORY 
UNITS 



!>590 

PROCESSOR 
UNITS 



DISK 

CONTROLLER 



502 500 
DISK INTERFACE 
NETWORK 



17/18 



FIG. 17 



HOST 

CHANNEL 

INTERFACE 



511 



511 



511 



511 



511 



511 



511 



511 



HOST HUB 



514 



603 



603 



HCA 



HCA 



18/18 



FIG. 18 



DISK 

CHANNEL 
INTERFACE 



521 



521 



604 



521 



521 



521 



521 



DISK HUB 



524 



521 



521 



525 



CACHE MEMORY 



604 



HCA 



HCA 



